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Abstract. Corresponding to n independent non-negative random variables X\, . . . , X n , 
are values Mi, . . . , M n , where each Mj is the expected value of the maximum of n inde- 
pendent copies of Xi. We obtain an upper bound to the expected value of the maximum 
of Xi, . . . , X n in terms of Mi, . . . , M n . This inequality is sharp in the sense that the 
quantity and its bound can be made as close to each other as we want. We also present 
. related comparison results. 

>>: 

I. Introduction 

To illustrate the main thrust of this paper, we consider a simple two-component parallel 
system; say a light pole made up of two light bulbs. If the system is considered to have 
p/ ' failed once both components, assumed to act independently, fail, then a reasonable measure 
Oh ■ of the performance of the system is E[max(X, Y)], where X and Y are the independent 
c-| ! random lifetimes of the two components. 

Assume that, to build the system, we may choose from any of two manufacturers (i.e. 
two lifetime distributions). Should we choose two components from the same manu- 
facturer or should we mix? In the case of manufacturers with identical performances 
(E[max(Xi, X 2 )] = E[max(Y 1; Y 2 )]), there is (almost) always a net gain in mixing. It is 
then natural to ask how much gain can one achieve and further to identify situations in 
which this gain is attained. 

In the spirit of the above example, we shall call a family of n independent random variables 
an n-assembly. When these are also identically distributed, we will say that they form a 
similar n-assembly. We call performance of an n-assembly (whether similar or not) the 
expected value of its maximum. The aim of this paper is to bound (from above as well as 
below) the performance of an n-assembly relying solely on the performances of all similar 
n-assemblies from which it is drawn. It will be shown that mixing (i.e. using assemblies 
issued from different distributions) improves performance by a factor, hereby called the 



X 

mixing factor, of up to (but not including) 2. We will further show that when all similar 
n-assemblies have the same performance, the mixing factor is at least 1. 
While an extensive literature exists on the expected value of the maximum of n independent 
and identically distributed random variables (see for example [3]), with the exemption of 
PQ and [6] (see also [2] and [31 Section 5.2]), not much work has concentrated on the case 
of non-identically distributed random variables. Furthermore, the aforementioned papers 
do not attempt a comparison with M l5 . . . , M n . In [T], the authors obtain upper and lower 
bounds in terms of E[XJ and var(A / " i ) (assumed to be finite) where X 1 , . . . , X n are possibly 
dependent random variables with possibly different distributions. These bounds generalise 
those of [I] and [5] which deal with the independent and identically distributed case. In 
[6], the author solely focuses on obtaining a lower bound. This is done by comparing 
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the distribution function of the maximum of X±, . . . ,X n to that of the maximum of n 
independent copies of an equally- weighted probability mixture of Xi, . . . ,X n (see later). 
More recently, in [7j, the author investigates the performance of an n-assembly constructed 
from 2 distributions and its behaviour as the make-up of the n-assembly changes. 

Notations and Assumptions. Throughout this paper, we use the following notations 
and assumptions. For any sequence Xi, . . . ,x n , we write X\ V x 2 V . . . V x n = Vfc=i x fc 
for the maximum max(xi, X2, ■ ■ ■ , x n ). Random variables will generally be indexed in the 
following way: X{ refers to the jth element in the similar n-assembly i; i.e. X} , . . . ,X" 
are independent random variables having the same distribution as Xi. = \l"l =l Xf, 

X (n) = VILi X u Mi = E[xl n) ], M (n) = V"=i Mi and M = ± ELi M t . Finally, we denote 
by 9 n the mixing factor of a set of n-assemblies: 

M {n) 

We assume that all random variables are non-negative and have finite first moment. 

2. The Mixing Factor Bounds 

2.1. The Main Results. The main theorem which we wish to show is: 

Theorem 1. Let Xi, . . . , X n be independent random variables and Mi = E[X f ], % = 
l,2,...,n. Then 

— —71 — 1 

(1) M < E[X {n) ] <M+ —^M {n) . 

In particular, if Mi = M, i = 1, ...,n, 

M < E[X (n) ] < (2- l/n)M. 



Corollary 2. For n-assemblies, the mixing factor does not exceed 2 — 1/n and, in the case 
of equally performing similar n-assemblies, it is at least 1. 

In the case where some of the random variables X\, . . . , X n are identically distributed, an 
improved upper bound may be achieved. Such a bound follows immediately from Theorem 
[Uwhen all distributions are repeated an equal amount. 

Corollary 3. Suppose the n-assembly X±, . . . ,X n is made up of k similar m-assemblies 
(n = km ). Then 

M < E[X {n) ] <M+ — ^— M (n) . 

Remark 4. Let Fi denote the distribution function of Xi and Gi = F™ be the distribution 
function of x\ . Then (QJ) can be rewritten as 

(2) < / (G(s) - G(s))ds < (1 - l/n) max / (1 - G t (s))ds, 

Jo i<i<"7o 

where G and G are the arithmetic and geometric means, respectively, of the distribution 
functions G±, G2, ■ ■ ■ , G n . 

Equation (TJ|) provides an upper bound (the lower bound is a direct consequence of the 
arithmetic and geometric mean inequality) on the L l -distance between the geometric and 
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arithmetic means of a sequence of n distribution functions on the positive half-line with 
finite means. 

In probabilistic terms, this expresses the following fact. Let Yi, . . . ,Y n be independent 
random variables (non-negative with finite mean). Let U be an equally-weighted probability 
mixture ofYi,...,Y n and V be such that the distribution of the maximum of n independent 
copies of V is that of F( n ) . Then 

< my] - E[U] < (1 - l/n) max E[F;]. 

l<i<n 

As hinted in the introduction, the lower bound is not optimal. In [H] (see also [2] and [3, 
Section 5.2]), it is shown that 

(3) P(X (n) < x) < P(Z (n) <x), Vx > 0, 

where Z l ,...,Z n are independent copies of an equally- weighted probability mixture of 
Xi, . . . , X n . Combining the inequality of the arithmetic and geometric means with a 
moment inequality, we can further bound the right hand side of 



(71 \ ^ 71 71 

-^>(X fe <x)j < -^P(X fe < x) n = -J2f(x^ ] < 
U k=l J U k=l U k=l 



X 



As a consequence, we obtain an improved lower bound for E[X( n 



M < E[Z (n) ] < E[X { 



Although a better bound for the performance of an n-assembly, E[Z^ n ^] cannot be expressed 
in terms of the performances of all similar n-assemblies from which it is drawn. In other 
words, EfZ*™)] is not a bound that can be expressed in terms of M 1; . . . , M n . Furthermore, 
it is shown in Corollary [T3] that, in the case of bounded random variables, an improved 
lower bound, expressed in terms of Mi,...,M n , can be achieved. This lower bound is 
shown, in the special case of a two-point distribution, to outperform E[Z^ n ^]. 

2.2. Two Toy Examples. Before we embark on the proof of the main Theorem, we look 
at the simple case of two-point distributions. The first example presented here will give 
some insight into how the improved bound of Corollary [T3] and that of [B] compare. The 
second example will demonstrate the sharpness of the upper bound of Theorem [TJ i.e. we 
show that E[X( n )] can be made as close to the upper bound as we want. 

(1) Assume that the random variables X±, . . . ,X n are all concentrated on two points, 
and b, and let pi = P[X, = 0]. Then the distribution of Z, the equally- weighted 

1 " 

probability mixture of X 1; . . . , X n , is given by W\Z = 0} = — } Pi = p and, 

n ^— ' 

i=l 

E[Z (n) ] = b(l - p n ) and M { = b(l - p?). 

Therefore, 

n n 

b - J] v^-M, - E[ZW] = bp n - b l[p t > 0, 

i=l i=l 

by a simple application of the inequality for the arithmetic and geometric means. 
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(2) Again, we assume that the random variables X±, . . . , X n are all concentrated on two 
points, or less. However, in this case, we allow the non-zero values x±, . . . , x n , to be 
different. In fact, we assume (without loss of generality) that X n is non-random, 
that all other random variables take 0, with probability pk = \/l — / Xk > 0, 
and Xk, with probability 1 — pk > 0, and that 

M(n-1) < M n < Xi < . . . < Xn-!. 

Then, it is shown in the proof of Proposition [8] (from which this very construction 
is extracted), that 



E[X (n _ 1} V M n ) = p n _ 1 E[X (n _ 2) V M n ] + (1 - p n _ x ) 



1 — n n 

1 Pn-l 



Letting p n -i go to 1 and x„_i to infinity so that M n _i remains constant, we see 

that E[X( n )] approaches E[X( n _ 2 ) V M n ] H — . Now letting p n -i go to 1 and 

%n-2 to infinity so that M n _ 2 remains constant, we see that E[X( n )] approaches 

E[X( n _ 3 ) VM n ]H 2 — — — — . Repeating this process leads to the fact that E[X( n )] 

approaches M n H ^— — — = M H Mi n \ (recall that M n = Mc n \). 

n n 

2.3. The Proofs. The lower bound is a direct consequence of the inequality for arithmetic 
and geometric means and is given here for completeness only. 

Proposition 5. If X\, . . . ,X n are independent random variables with the property that 



en 



E[Xf'} = Mi, i = 1,2, ...,n , then E[X (n) ] > M. Furthermore, if E[X (n) ] = M th 
X\, . . . , X n are identically distributed. 

Proof. Because the arithmetic mean is greater than the geometric mean, 

1 " 

(4) l-U X U 2 ...U n > - V(l-<). 

i=i 

This implies 

/"OO -I n poo 

E[X (n) ] = / [1 - F 1 (x)F 2 (x)...F n (x)]dx > - V / [1 - i^ ?i (x)]dx = M, 

where Fi denotes the distribution function of X^. Further, since (jlj) turns into an equality 
if and only if the u^s are all equal, the lower bound of E[X( n )] is only attained when 
Fi — . . . — F n , that is when X\, . . . , X n are identically distributed. □ 

The first step in the proof of the upper bound is to reduce the problem to the case of 
random variables concentrated on a finite set of points. This is easily demonstrated by 
using the approximation 

"m2" 



X = lim 

mtoo 



~^T 1 [('- 1 )/ 2m .'/2 m ) ( x ) + ml[ m , + oo)P0 



. 1=1 

Indeed, one only needs to apply the Monotone Convergence Theorem to prove the following 
proposition. 
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Proposition 6. If TheoremUl is true for random variables concentrated on a finite set of 
points, then it must be true for general random variables. 

Having reduced the problem to one that only involves random variables concentrated 
on a finite set of points, we shall find the maximum possible value of E[X( n )] when we 

fix E[X^] = Mi, i = l,2,...,n, by continually creating new sets of n-assemblies which 
maintain the property ELY^] = M,, but increase, or at least do not decrease, E[X( n d. 
The main step in achieving the upper bound announced in Theorem [1] is to prove that 
the problem can be reduced to one that involves random variables that take exactly 
one non-zero value. This is done by showing that, for any random variable, two non- 
zero adjacent points can be coalesced into a single point; that is, a random variable 
which takes values x±, . . . ,x r (x± < ... < x r ) with probabilities pi, . . . ,p r respectively, 
can be replaced by a random variable with masses pi, ■ ■ ■ ,pi-i,pi + pi + i,pi + 2, ■ ■ ■ ,p r at 
x±, . . . , Xi-i, x, Xi + 2, . . . ,x r , for a carefully chosen x. This is initially done in Proposition 
[9] for adjacent points not separated by points from other random variables, and later ex- 
tended to the general case in Proposition [10] yielding the following Theorem. 

Proposition 7. // TheoremU\is true for random variables which take exactly one non-zero 
value, then it must be true for general random variables. 

Proposition 8. Let X±, . . . ,X n be independent random variables with the property that 
^[X^] = M it % = 1,2, ...,n. If each of Xi, . . . ,X n takes exactly one non-zero value, then 

— — Ti — 1 

M < E[X {n) ] < M + M {n) . 



Proof. For k e {1, . . . , n}, we denote by x^ the non-zero value of X^ and pk = F[Xk = 0] . 
The next four points successively simplify the problem. 

(1) If W[Xk = 0] > 0, for all k (i.e. all X^s place a positive mass at 0), then applying 
Proposition [9] to X\ (assumed without loss of generality to have the smallest non- 
zero value) and Y = \J^ =2 Xk shows that E[X( n )] is increased if X\ is replaced by 

Mi = E[x{ n) ]. Therefore we may assume that at least one of the X^s is constant. 

(2) Without loss of generality, we may assume that X n is the largest constant random 
variable. X n may be one of any number of constant random variables, all equal, 
that are larger than any other constant random variable. Assume that at least one 
constant random variable is strictly smaller than X n . Let it be a (a < X n ). Since 
E[a V X n V Y] = E[Z Vl„V Y], where Z is such that F[Z = 0] = (1 - a/X n ) l ' n 
and F[Z = X n ] = 1 - ( 1 - a /X n ) l / n (a V X n = X V X n = X n ), we may assume that 
all X^s satisfying M^ < M n , are non-constant (place a positive mass on 0). 

(3) If one other X k , say X n _ ly satisfies X n _i = M n _i = M n , then replacing X n _i by Y 
such that P[y = 0] = q > 0, P[Y = y] = 1 - q and y = M n /(1 - q n ) > M n , we get 

E[X (n _ 2) V Y V M n ] 
= gE[X (n _ 2) V M n ] + (1 - g)E[X (n _ 2) V y V M„] 
> E[X (n _ 2) VM n ] =E[X (n) ]. 

Therefore we may assume that the random variable with the largest M& (say M n ) 
is constant and all others are non-constant. 
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(4) Without loss of generality, we may assume that x± < ... < £ n -i. Recall that, 
for k G {1, . . . ,n — 1}, M k = (1 — pl)x k . If x\ < M n then, with Y such that 
P[y = 0] = q > 0, F[Y = y] = 1 - q and y = M 1 /{1 - q n ) > M n , 

E[Y VZV M„] = gE[Z V M n ] + (1 - g)E[y VZV M n ] > E[X (n) }, 
where Z = V&=2 -^fc- Therefore we may assume that M( n _ 1 ) < M n < x\ < . . . < 

Xn—l- 



Now, 



E[X (n) ] = E[X {n _ 2) V X n _ x V M n ] 

= p n _iE[X (n _ 2 ) V M n ] + (1 - 



= p n _!E[X (n _ 2) V M n ] + (1 - p n _ x ) 

Next we use the fact that the function (j)(p) = up + v jEpK defined on [0, 1] (extended at 1 
by continuity), where v < u, increases from v to u + v/n. Since E[X( n _ 2 ) V M n ] > M n _\, 
we immediately get that 



E[X (n) ] < E[X [n _ 2) V M n ] 



Repeating the same argument, we get 



EI(n) < E X (n _ 3) V M n + < ■■■< M n + . 

n n 

□ 

3. Comparison Results 

In this section we develop the tools required to obtain the bounds of the previous section. 
However, these tools are important in their own right. They enable us to increase the 
performance of an n-assembly while keeping the performances of all similar n-assemblies 
unchanged. We prove them for random variables that are not necessarily concentrated 
on a finite set of points. We also obtain a comparison result that allows to decrease the 
performance of an n-assembly while keeping the performances of all similar n-assemblies 
unchanged. 

Proposition 9. // Xi and Y are independent, P[a < Xi < b] = p and P[a < Y < b] = 

then we may replace X\ by a random variable X 2 such that X 2 = X\ outside [a, b] and, 
P[a < X 2 < b] = F[X 2 = x] = p, for some a < x < b with the property that E[X 2 nS> ] = 
E[x[ n) ] and E[X 2 V Y) > E[X t V Y]. 

Proof. Equating the contributions to E[x[ n ^] and EfXg' 1 ''] from the interval [a, b] gives 

x{F(b) n - F~(a) n ) = (f\lJ s F-{a) n - k E[xl k) \a < X{ < b, j = 1, . . . , k] 

k=i ^ ' 

where F is the distribution function of Xi and F~(x) = lim^^ F(y). We thus see that x is 
a convex combination of the n expectations on the right hand side and thus, in particular, 
that x > min 1 < fc < n E[xf ) |a < X 1 < b] = E[X x |a < X x < b}. Finally 

E[X 2 V Y] - E[Xi V Y] = p¥[Y < a](x - E[Xi|a < X 1 < b]) 
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and, as previously observed, the second expression is nonnegative. □ 

If we allow P[a < Y < b] > then the coalescing point is no longer necessarily in the 
interval [a, b] as is demonstrated in the next result. 

Proposition 10. Assume that X\ and Y are independent and that the entire mass X\ 
places on an interval (l,r) is concentrated on two values a and b within it: < I < a < 
b < r, P[Z < X 1 < r] = F[X X e {a, b}}, p = F[X X = a] > and q = PLY X = b) > 0. Then 
there exists a random variable X 2 s.t. E[x[ n) ] = E[X^ n) ], E[X 1 V Y] < E[X 2 V Y] and the 
mass X 2 places on the interval (I, r) is concentrated on at most one single value within it. 

Proof. Let X 2 = X 1 l Xl <i + ulx 1=a + V{u)lx 1= b + Xil Xl > r , where u is a free parameter, 

V(u) = b — \ n (u — a) and A n = — — . . 

v ; ny ) n F(b) n -F(a) n 

Then E[X^ n) ] = E[x{ n) ] and E[X 2 VY] — E[X X V Y] + <j>{u) - 0(a), where 

(f)(u) = pE[u V Y] + qE[V(u) V Y]. 

Using Lemma [H] below, we get that <f>' + (u) = pG{u) — q\ n G-(V(u)) is non-decreasing and 
that 0+(a) = pG(a) — q\ n G-{b) is not zero unless G(a) > (since G-(b) > P[a < Y < 
b] > 0). 

Therefore there are three possible scenarios, either <f>' + (a) < 0, <f>' + (a) > or, 4>' + (a) = 
and G(a) > 0. In the first case, E[X 2 V Y] decreases on [I, a] and letting u = I achieves the 
objective. In the second case, E[X 2 V Y] increases on [a,r] and it suffices to let u = r. 
Finally, suppose <f/ + (a) = and G(a) > 0. Let u = E[x[ n) \l < x[ n) < r]. Then a < u < b, 
V{uS) = uj and 

n-l 

<KM > G ^ F{b) n n F{a) n E F « [Fib) M ~ F{lT- k ^] > 0. 

Here it suffices to let u = uj (a and b are merged into uj). 

□ 

Lemma 11. For any positive random variable Y with distribution function G and differ- 
entiable function h, 7(11) = E[h(u) V Y] admits left and right derivatives (that differ on a 
set that is at most countable): 

, ( s _ / h'{u)G ± (h{u)) if h\u) > 
j ± [u) - I h >fa G ^ h ( u fi ^ h >( u ) < 

The next proposition enables the reduction of E[X( n )] for given Mi, i = 1, . . . , n. 

Proposition 12. Let b be such that ¥[Xi < b] > 0, for all i. Then, for any interval 
I = [a,b], there are random variables Y\, . . . ,Y n such that E[X( n )] > E[Y( n )] and, for all 

i = l,...,n,Yi = Xi on{Xi& I}, P[a < Yi < b] = and E[X^ n) ] = E[Y} n) }. 

Proof. Let i — 1, . . . , n, be such that P[& = a] = a i: P[^ = b] — 1 — a, and ^1, . . . , £ n 
are independent of each other and of all other random variables Xj. Now let 

Yi = £ilxiei + Xil Xi 
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and form the corresponding similar n-assemblies, (Y^ 1 , . . . , 3 / i n )i=i,..., w . Our first objective 
is to select a, such that E[X^] = E[Y"/ n ^]. For simplicity, we shall momentarily drop the 
index % and compute more generally E[X( n d — E[Y7 n )]. Further, we introduce the notation, 
z = zVo - a = (z — a) + and observe that, 

(1) a < z < b iff z > a and z < b — a 

(2) Zfa) = Z(n) 

(3) if /C is set of non-empty subsets of N = {1, . . . , n}, 



(5) z {n) Yi iz k > a = z {n) ~ j2 v ik ) n iz ^ j ) n i *k<« 

k=l K&K. \k£K J \kGK J \k<£K 

For KeK, let 

A K =f) {X k G /}, B K = p| {X k < a}, C K = A K n B K 

k&K k£K 

and for any sequence zi, . . . , z n , z\k\ = Vfceit Zk - Using the identities, 

{X( n) el}= [j C K and X (n) - Y"( n) = ^ (^[a-] - £[k])1c k : 



Keic KeK. 



we obtain 



E[X (n) ] - E[Y {n) ] 



^ (e[X [x] + a, Ck] - E[eV] + a, c7 A 
KeK. 

P(S*) (e[X [a1 , A*] - E[i [K] , A K 



K&K. 



E[X (n) , A N ] + Yl HB K )E[X lK] , A K ] - V(A K )F(B K )E[£ m ) 

KeK\{N} KeK. 

E[x {n) ,D N }- e ^ik] > d n n n 

i^e/c\{7V} 

+ ^ P( J B A -)E[X^,A i ,]-^P(^)P( J B / ,)E[e [A1 ] 
A'e/c\{7V} tfe/c 
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where D K = f] keK {X k < b - a}, E K = f] keK {Xk > a} and we have used (jSJ). Applying 
the identities PI -Br- = -D^ n -B^ and fl Ek = Ak, it follows that 



E[X (n) ] - E[Y (n) ] 

= E[X (n) ,Av]- ^ P( J B x )E[X [x] ,A i ,] 
Ke/C\{iV} 

+ Yl nB K )E[X [K] , A K ] - nA K )¥(B K )E[i [K] 

K£K\{N} KelC 

= E[X (n) ,D N ] - nA K )F(B K )E[i [K] ] 



(6) 



Keic 

-Hi _ nrrvHi 



Therefore, with Pi = F[X { e I], E[Xr } ] = E[F> nj ] if and only if 

(ar- m (l-aT)(b-a) 



E[xt\x^<b- a \ = ir( n XrF> 

m=l ^ ' 

m=l ^ ' 



(6 -a) 



m=l ^ ' 



(6-a)[F 4 (fe) n -(F-(a)+^r] 



that is 
(7) 



F i (a) + 



Returning to (jSJ), we find 



(b - a) = (b - a) - E[xl n) \xl n) < b - a}. 



E[X (n) ] - E[Y (n) ] 
= E[X (n) , D N ] - nA K )F(B K )E[i [K] ] 



K&K. 



E[x in) , x in) < b - a\ - e n p* n ( i - n ^ ) ( & 

KeKkeK k^K \ fcG-ftT / 



E[X {n) \X (n) <b-a]l[F k (b) 



k=l 



\F k (b)- \F-{a)- -\{F-{a)+ Pk a k ) + \\F k 



k=l 



k=l 



k=l 



k=l 



(b-a) 



E[X {n) \X {n) <b-a]Y[F k (b) 



k=l 



\F k (b)- -\{F k -{a)+ Pk a k ) 



.k=l 



k=l 



(b-a) 



\F k (b) I E[X {n) \X {n) <b-a]-(b-a) + (b-a)\ 



k=i 



k=i 



F k ( Q ) + Pk®k 

Fkib) 
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Now, by a simple application of Holder's inequality, we get 

(pb—a n 
n r b-a 



< n / \i-G k { Z )) n dz 
k=i j ° 

n 

l[({b-a)-nxP\xP<b-a] 
nA ( F k( a ) +Pkak Y 



k=l 



(6 -a) 



where Gk is the conditional distribution function of {b — a) — Xk given {Xk <b — a} and 
we have used (j2J). It immediately follows (recall that (b — a) — K[Xj^\xj^ < b — a] > 0) 
that 

(6 - a) - E[X (n) |X (n) < b - a] < (b - a) f[ ( ^(^ ) 

which completes the proof. □ 

A consequence of this proposition is an improved lower bound in the case of bounded 
random variables. 

Corollary 13. If X ly . . . ,X n are independent bounded random variables with the property 
that E[xf n) ] = Mi,i = 1, 2, n , then 



E[X (B) ]>6-] [Vb-Mi>M, 



i=l 

where b is a common upper bound to all Xj 's. 

Proof. Applying Proposition [T2] to the interval / = [0, b] and the random variables such 
that F[Y { = 0] = 1 - F[Yi = b] = y 1 - — ^, we immediately get that 



E[A (n) ] > E[y (n) ] = (i-nv 1- Tr 6 ~ri v 76 - M * ^ m. 

\ i=l / i=l 

The second inequality is a direct consequence of the inequality for arithmetic and geometric 
means. □ 
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